Good Statistical Practice-development of tailored Good Clinical Practice training for statisticians.

BACKGROUND: Statisticians are fundamental in ensuring clinical research, including clinical trials, are conducted with quality, transparency, reproducibility and integrity. Good Clinical Practice (GCP) is an international quality standard for the conduct of clinical trials research. Statisticians are required to undertake training on GCP but existing training is generic and, crucially, does not cover statistical activities. This results in statisticians undertaking training mostly unrelated to their role and variation in awareness and implementation of relevant regulatory requirements with regards to statistical conduct. The need for role-relevant training is recognised by the UK NHS Health Research Authority and the Medicines and Healthcare products Regulatory Agency (MHRA). METHODS: The Good Statistical Practice (GCP for Statisticians) project was instigated by the UK Clinical Research Collaboration (UKCRC) Registered Clinical Trials Unit (CTU) Statisticians Operational Group and funded by the National Institute for Health and Care Research (NIHR), to develop materials to enable role-specific GCP training tailored to statisticians. Review of current GCP training was undertaken by survey. Development of training materials were based on MHRA GCP. Critical review and piloting was conducted with UKCRC CTU and NIHR researchers with comment from MHRA. Final review was conducted through the UKCRC CTU Statistics group. RESULTS: The survey confirmed the need and desire for the development of dedicated GCP training for statisticians. An accessible, comprehensive, piloted training package was developed tailored to statisticians working in clinical research, particularly the clinical trials arena. The training materials cover legislation and guidance for best practice across all clinical trial processes with statistical involvement, including exercises and real-life scenarios to bridge the gap between theory and practice. Comprehensive feedback was incorporated. The training materials are freely available for national and international adoption. CONCLUSION: All research staff should have training in GCP yet the training undertaken by most academic statisticians does not cover activities related to their role. The Good Statistical Practice (GCP for Statisticians) project has developed and extensively piloted new, role-specific, comprehensive, accessible GCP training tailored to statisticians working in clinical research, particularly the clinical trials arena. This role-specific training will encourage best practice, leading to transparent and reproducible statistical activity, as required by regulatory authorities and funders.

Statistical recommendations for count, binary, and ordinal data in rare disease cross-over trials.

BACKGROUND: Recommendations for statistical methods in rare disease trials are scarce, especially for cross-over designs. As a result various state-of-the-art methodologies were compared as neutrally as possible using an illustrative data set from epidermolysis bullosa research to build recommendations for count, binary, and ordinal outcome variables. For this purpose, parametric (model averaging), semiparametric (generalized estimating equations type [GEE-like]) and nonparametric (generalized pairwise comparisons [GPC] and a marginal model implemented in the R package nparLD) methods were chosen by an international consortium of statisticians. RESULTS: It was found that there is no uniformly best method for the aforementioned types of outcome variables, but in particular situations, there are methods that perform better than others. Especially if maximizing power is the primary goal, the prioritized unmatched GPC method was able to achieve particularly good results, besides being appropriate for prioritizing clinically relevant time points. Model averaging led to favorable results in some scenarios especially within the binary outcome setting and, like the GEE-like semiparametric method, also allows for considering period and carry-over effects properly. Inference based on the nonparametric marginal model was able to achieve high power, especially in the ordinal outcome scenario, despite small sample sizes due to separate testing of treatment periods, and is suitable when longitudinal and interaction effects have to be considered. CONCLUSION: Overall, a balance has to be found between achieving high power, accounting for cross-over, period, or carry-over effects, and prioritizing clinically relevant time points.

Limitation of permutation-based differential correlation analysis.

The comparison of biological systems, through the analysis of molecular changes under different conditions, has played a crucial role in the progress of modern biological science. Specifically, differential correlation analysis (DCA) has been employed to determine whether relationships between genomic features differ across conditions or outcomes. Because ascertaining the null distribution of test statistics to capture variations in correlation is challenging, several DCA methods utilize permutation which can loosen parametric (e.g., normality) assumptions. However, permutation is often problematic for DCA due to violating the assumption that samples are exchangeable under the null. Here, we examine the limitations of permutation-based DCA and investigate instances where the permutation-based DCA exhibits poor performance. Experimental results show that the permutation-based DCA often fails to control the type I error under the null hypothesis of equal correlation structures.

The Lure of Beauty: People Select Representations of Statistical Information Largely Based on Attractiveness, Not Comprehensibility.

OBJECTIVE: People differ in whether they understand graphical or numerical representations of statistical information better. However, assessing these skills is often not feasible when deciding which representation to select or use. This study investigates whether people choose the representation they understand better, whether this choice can improve risk comprehension, and whether results are influenced by participants' skills (graph literacy and numeracy). METHODS: In an experiment, 160 participants received information about the benefits and side effects of painkillers using either a numerical or a graphical representation. In the "no choice" condition, the representation was randomly assigned to each participant. In the "choice" condition, participants could select the representation they would like to receive. The study assessed gist and verbatim knowledge (immediate comprehension and recall), accessibility of the information, attractiveness of the representation, as well as graph literacy and numeracy. RESULTS: In the "choice" condition, most (62.5%) chose the graphical format, yet there was no difference in graph literacy or numeracy (nor age or gender) between people who chose the graphical or the numerical format. Whereas choice slightly increased verbatim knowledge, it did not improve gist or overall knowledge compared with random assignment. However, participants who chose a representation rated the representation as more attractive, and those who chose graphs rated them as more accessible than those without a choice. LIMITATIONS: The sample consisted of highly educated undergraduate students with higher graph literacy than the general population. The task was inconsequential for participants in terms of their health. CONCLUSIONS: When people can choose between representations, they fail to identify what they comprehend better but largely base that choice on how attractive the representation is for them. HIGHLIGHTS: People differ systematically in whether they understand graphical or numerical representations of statistical information better. However, assessing these underlying skills to get the right representation to the right people is not feasible in practice. A simple and efficient method to achieve this could be to let people choose among representations themselves.However, our study showed that allowing participants to choose a representation (numerical v. graphical) did not improve overall or gist knowledge compared with determining the representation randomly, even though it did slightly improve verbatim knowledge.Rather, participants largely chose the representation they found more attractive. Most preferred the graphical representation, including those with low graph literacy.It would therefore be important to develop graphical representations that are not only attractive but also comprehensible even for people with low graph literacy.

"Selection of statistical tests influences the interpretation of a study outcome" a comment on "Giriraja KV, Bhatnagar SK, Tomlinson L, Sancilio F. An open-label, multicenter, phase 2 study of a food enriched with docosahexaenoic acid in adults with sickle cell disease. Prostaglandins Leukot Essent Fatty Acids. 2023 Jun; 193:102574. Doi: 10.1016/j.plefa.2023.102574. Epub 2023 Apr 7. PMID: 37121179".

Parametric tests such as t-tests require a normal distribution of data. However, the determination of normal distribution may not be conclusive while dealing with a small sample size. Non-parametric tests such as Wilcoxon tests may be used in this situation, as these tests do not require normal distribution.

Nonparametric item response models: a comparison on recovering true scores / Modelos de resposta ao item não-paramétricos: comparando a recuperação de escores verdadeiros / Modelos de respuesta al item no-paramétricos: comparando la recuperación de puntuaciones verdaderas

Nonparametric procedures are used to add flexibility to models. Three nonparametric item response models have been proposed, but not directly compared: the Kernel smoothing (KS-IRT); the Davidian-Curve (DC-IRT); and the Bayesian semiparametric Rasch model (SP-Rasch). The main aim of the present study is to compare the performance of these procedures in recovering simulated true scores, using sum scores as benchmarks. The secondary aim is to compare their performances in terms of practical equivalence with real data. Overall, the results show that, apart from the DC-IRT, which is the model that performs the worse, all the other models give results quite similar to those when sum scores are used. These results are followed by a discussion with practical implications and recommendations for future studies.(AU)

Procedimentos não paramétricos são usados para adicionar flexibilidade aos modelos. Três modelos não paramétricos de resposta ao item foram propostos, mas não comparados diretamente: o Kernel smoothing (KS-IRT); a Curva Davidiana (DC-IRT); e o modelo semiparamétrico Rasch Bayesiano (SP-Rasch). O objetivo principal do presente estudo é comparar o desempenho desses procedimentos na recuperação de escores verdadeiros simulados, utilizando escores de soma como benchmarks. O objetivo secundário é comparar seus desempenhos em termos de equivalência prática com dados reais. De forma geral, os resultados mostram que, além do DC-IRT, que é o modelo que apresenta o pior desempenho, todos os outros modelos apresentam resultados bastante semelhantes aos de quando se usam somatórios. Esses resultados são seguidos de uma discussão com implicações práticas e recomendações para estudos futuros.(AU)

Se utilizan procedimientos no paramétricos para agregar flexibilidad a los modelos. Se propusieron tres modelos de respuesta al ítem no paramétricos, pero no se compararon directamente: Kernel smoothing (KS-IRT); la curva davidiana (DC-IRT); y el modelo bayesiano de Rasch semiparamétrico (SP-Rasch). El objetivo principal del presente estudio es comparar el desempeño de estos procedimientos en la recuperación de puntajes verdaderos simulados, utilizando puntajes de suma como puntos de referencia. El objetivo secundario es comparar su desempeño en términos de equivalencia práctica con datos reales. En general, los resultados muestran que, a excepción de DC-IRT, que es el modelo con peor desempeño, todos los otros modelos presentan resultados bastante similares a los obtenidos cuando se utilizan sumatorios. Estos resultados son seguidos por una discusión con implicaciones prácticas y recomendaciones para estudios futuros.(AU)

[Statistical power in medical research. What stance should be taken when the research results are significant?] / Potencia estadística en investigación médica. ¿Qué postura tomar cuando los resultados de la investigación son significativos?

When researchers request funding and authorisation from financial institutions to carry out their project, one of the first questions they are asked is: what is the statistical power of the study you are proposing? If the researcher answers, for example, 90%, and the evaluator is satisfied, it is certain that he/she is not really familiar with the subject. The power of a study is not unique. It depends on certain parameters and what happens is that, in most cases, by introducing a slight variation in the values of these parameters, the power takes on an acceptable value. If this is not the case and the study is carried out anyway, and its results are very significant, there is no room to question its success by arguing that the power of the study was very low. It is just the time to celebrate.

TITLE: Potencia estadística en investigación médica. ¿Qué postura tomar cuando los resultados de la investigación son significativos?Cuando el investigador pide subvención y autorización a entidades financieras para llevar a cabo su proyecto, entre las primeras cuestiones que le plantean está: ¿qué potencia estadística tiene este estudio que usted propone? Si el investigador responde, por ejemplo, el 90%, y el evaluador se da por satisfecho, es seguro que no conoce realmente el tema. La potencia de un estudio no es única. Depende de determinados parámetros y ocurre que, en la mayoría de los casos, variando ligeramente los valores de esos parámetros, la potencia toma un valor aceptable. Si no es así, y a pesar de ello se lleva a cabo el estudio, y sus resultados son muy significativos, no ha lugar a cuestionar el éxito encontrado argumentando que el estudio tenía poca potencia. Tan sólo es momento de celebrarlo.

A corrected smoothed score approach for semiparametric accelerated failure time model with error-contaminated covariates.

We consider the semiparametric accelerated failure time (AFT) model with multiple covariates measured with error. Existing methods for the AFT model are either inconsistent, computationally intensive, or require stringent assumptions. To overcome these limitations, we develop a correction approach for a general smooth function of error-contaminated variables. We apply this method to the smoothed rank-based score function for the AFT model. The estimator is consistent and asymptotically normal. The finite-sample performance of the method is assessed by simulation studies. The approach is illustrated by application to data from an HIV clinical trial.

Leading beyond regulatory approval: Opportunities for statisticians to optimize evidence generation and impact clinical practice.

The role and value of statistical contributions in drug development up to the point of health authority approval are well understood. But health authority approval is only a true 'win' if the evidence enables access and adoption into clinical practice. In today's complex and evolving healthcare environment, there is additional strategic evidence generation, communication, and decision support that can benefit from statistical contributions. In this article, we describe the history of medical affairs in the context of drug development, the factors driving post-approval evidence generation needs, and the opportunities for statisticians to optimize evidence generation for stakeholders beyond health authorities in order to ensure that new medicines reach appropriate patients.

Group sequential testing under instrumented difference-in-differences approach.

Unmeasured confounding is a major obstacle to reliable causal inference based on observational studies. Instrumented difference-in-differences (iDiD), a novel idea connecting instrumental variable and standard DiD, ameliorates the above issue by explicitly leveraging exogenous randomness in an exposure trend. In this article, we utilize the above idea of iDiD, and propose a novel group sequential testing method that provides valid inference even in the presence of unmeasured confounders. At each time point, we estimate the average or conditional average treatment effect under iDiD setting using the data accumulated up to that time point, and test the significance of the treatment effect. We derive the joint distribution of the test statistics under the null using the asymptotic properties of M-estimation, and the group sequential boundaries are obtained using the α $$ \alpha $$ -spending functions. The performance of our proposed approach is evaluated on both synthetic data and Clinformatics Data Mart Database (OptumInsight, Eden Prairie, MN) to examine the association between rofecoxib and acute myocardial infarction, and our method detects significant adverse effect of rofecoxib much earlier than the time when it was finally withdrawn from the market.

Approaching the body of evidence: Key concepts of Overviews. / Abordando el cuerpo de evidencia: conceptos fundamentales de los overviews.

The increasing production of primary research and literature reviews in the last decades has made it necessary to develop a new methodological design to synthesize the evidence: the overviews. An overview is a type of evidence synthesis that uses systematic reviews as the unit of analysis, with the aim of extracting and analyzing the results for a new or broader research question, helping the shared decision-making processes. The aim of this article is to introduce the reader to this type of evidence summaries, highlighting the differences between overviews and other types of synthesis, the unique methodological aspects of overviews, and future challenges. This is the twelfth article from a collaborative methodological series of narrative reviews about biostatistics and clinical epidemiology.

El aumento de la producción de investigación primaria y de las revisiones de la literatura durante las últimas décadas ha hecho necesario el desarrollo de un nuevo diseño metodológico para sintetizar la evidencia: los overviews. Un overview es un diseño de síntesis de evidencia que toma como unidad de análisis a las revisiones sistemáticas, con el objetivo de extraer y analizar los resultados para una pregunta de interés nueva o más amplia, ayudando así a mejorar los procesos de toma de decisiones informadas. El objetivo de este artículo es introducir al lector a este tipo de resúmenes de evidencia, destacando las diferencias con los otros tipos de síntesis de evidencia, los aspectos metodológicos particulares de los overviews, y los desafíos pendientes. Este artículo es el duodécimo de una serie metodológica colaborativa de revisiones narrativas sobre temáticas de bioestadística y epidemiología clínica.

Multiple multi-sample testing under arbitrary covariance dependency.

Modern high-throughput biomedical devices routinely produce data on a large scale, and the analysis of high-dimensional datasets has become commonplace in biomedical studies. However, given thousands or tens of thousands of measured variables in these datasets, extracting meaningful features poses a challenge. In this article, we propose a procedure to evaluate the strength of the associations between a nominal (categorical) response variable and multiple features simultaneously. Specifically, we propose a framework of large-scale multiple testing under arbitrary correlation dependency among test statistics. First, marginal multinomial regressions are performed for each feature individually. Second, we use an approach of multiple marginal models for each baseline-category pair to establish asymptotic joint normality of the stacked vector of the marginal multinomial regression coefficients. Third, we estimate the (limiting) covariance matrix between the estimated coefficients from all marginal models. Finally, our approach approximates the realized false discovery proportion of a thresholding procedure for the marginal p-values for each baseline-category logit pair. The proposed approach offers a sensible trade-off between the expected numbers of true and false findings. Furthermore, we demonstrate a practical application of the method on hyperspectral imaging data. This dataset is obtained by a matrix-assisted laser desorption/ionization (MALDI) instrument. MALDI demonstrates tremendous potential for clinical diagnosis, particularly for cancer research. In our application, the nominal response categories represent cancer (sub-)types.

Multivariate single index modeling of longitudinal data with multiple responses.

In medical studies, composite indices and/or scores are routinely used for predicting medical conditions of patients. These indices are usually developed from observed data of certain disease risk factors, and it has been demonstrated in the literature that single index models can provide a powerful tool for this purpose. In practice, the observed data of disease risk factors are often longitudinal in the sense that they are collected at multiple time points for individual patients, and there are often multiple aspects of a patient's medical condition that are of our concern. However, most existing single-index models are developed for cases with independent data and a single response variable, which are inappropriate for the problem just described in which within-subject observations are usually correlated and there are multiple mutually correlated response variables involved. This paper aims to fill this methodological gap by developing a single index model for analyzing longitudinal data with multiple responses. Both theoretical and numerical justifications show that the proposed new method provides an effective solution to the related research problem. It is also demonstrated using a dataset from the English Longitudinal Study of Aging.

Machine Learning and Statistics in Clinical Research Articles-Moving Past the False Dichotomy.

This Viewpoint describes the false dichotomy between statistics and machine learning and suggests considerations in building and evaluating clinical prediction models.

Statistical learning methods and cross-cultural fairness: Trade-offs and implications for risk assessment instruments.

The use of statistical learning methods has recently increased within the risk assessment literature. They have primarily been used to increase accuracy and the area under the curve (AUC, i.e., discrimination). Processing approaches applied to statistical learning methods have also emerged to increase cross-cultural fairness. However, these approaches are rarely trialed in the forensic psychology discipline nor have they been trialed as an approach to increase fairness in Australia. The study included 380 Aboriginal and Torres Strait Islander and non-Aboriginal and Torres Strait Islander males assessed with the Level of Service/Risk Needs Responsivity (LS/RNR). Discrimination was assessed through the AUC, and fairness was assessed through the cross area under the curve (xAUC), error rate balance, calibration, predictive parity, and statistical parity. Logistic regression, penalized logistic regression, random forest, stochastic gradient boosting, and support vector machine algorithms using the LS/RNR risk factors were used to compare performance against the LS/RNR total risk score. The algorithms were then subjected to pre- and postprocessing approaches to see if fairness could be improved. Statistical learning methods were found to produce comparable or marginally improved AUC values. Processing approaches increased several fairness definitions (namely xAUC, error rate balance, and statistical parity) between Aboriginal and Torres Strait Islanders and non-Aboriginal and Torres Strait Islanders. The findings demonstrate that statistical learning methods may be a useful approach to increasing the discrimination and cross-cultural fairness of risk assessment instruments. However, both fairness and the use of statistical learning methods encompass significant trade-offs that need to be considered. (PsycInfo Database Record (c) 2023 APA, all rights reserved).